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(Cost  Considerations  in  Systems  Analysis) 

•kit 
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The  RAND  Corporation,  Santa  Monica,  California 


INTRODUCTION 


The  President  of  the  United  States  held  a  news  conference  on  Au¬ 
gust  25,  1965.  One  of  the  most  significant  things  he  said  was  the 
following: 

This  morning  I  have  just  concluded  a  breakfast  meeting  with 
the  Cabinet  and  with  the  heads  of  Federal  agencies  and  I  am 
asking  each  of  them  to  immediately  begin  to  introduce  a  very 
new  and  very  revolutionary  system  of  planning  and  program¬ 
ming  and  budgeting  throughout  the  vast  Federal  Government, 
so  that  through  the  tools  of  modem  management  the  full  prom¬ 
ise  of  a  finer  life  can  be  brought  to  every  American  at  the 
lowest  possiblf  cost.^ 

The  era  of  program  budgeting  had  begun  for  the  entire  Federal  Gov¬ 
ernment.  In  effect  what  the  President  said  was:  "I  want  all  depart¬ 
ment  and  agency  heads  to  try  to  do  what  McNamara  and  Hitch  hive  been 
attempting  in  Defense  since  1961.” 

Even  in  1961  program  budgeting  was  not  new--at  least  in  basic  con¬ 
cept.  It  had  been  proposed  for  the  Department  of  Defense  as  early  as 
1953.*  And  prior  to  that  something  akin  to  program  budgeting  had  been 


A  draft  paper  to  be  presented  to  a  conference  on  the  Economic 
Analysis  of  Public  Products  at  Princeton  University  in  April  1968. 

The  conference  is  to  be  sponsored  by  the  Universities-National  Bureau 
of  Economic  Research  Committee  on  Production  and  Distribution  of  Pub¬ 
lic  Products. 

Any  views  expressed  in  this  paper  are  those  of  the  author.  They 
should  not  be  interpreted  as  reflecting  the  views  of  The  RAN”  Corpora¬ 
tion  or  the  official  opinion  or  policy  of  any  of  its  govemm  ntal  or 
private  research  sponsors.  Papers  are  reproduced  by  The  RAND  Corpora¬ 
tion  as  a  courtesy  to  members  of  its  staff. 

*The  New  York  TimeB .  August  26,  1965. 

David  Novick,  Efficiency  and  Economy  in  Government  Through  New 
Budgeting  and  Accounting  Procedures,  R-254  (Santa  Monica,  Calif.:  The 
RAND  Corporation,  1953). 
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used  in  the  War  Production  Board  in  World  War  II.  Its  use  in  industry 

irk 

apparently  date*  back  to  at  least  1924. 

But  in  spite  of  this  history,  the  President's  announcement  in  the 
summer  of  1965  did  pose  something  of  a  "revolution"  for  many  departments 

and  agencies  in  the  Federal  Government.  In  terms  of  the  ultimate  goal, 
planning,  programing  and  budgeting  would  have  to  become  an  integrated 
process- -aome thing  which  (surprisingly)  was  not  generally  true  at  thaL 
time.  The  "pure  requirements"  approach  to  planning  would  have  to  be 
modified.  Staunch  advocates  of  new  programs  could  no  longer  argue  ef¬ 
fectively  in  terms  of:  "We  need  new  program  X  because  there  is  a  re¬ 
quirement  for  it;  there  is  a  requirement  because  we  need  it";  and  so  on. 

In  the  planning  process,  alternatives  would  have  to  be  examined 
systematically,  subject  to  realistic  resource  constraints.  And  alter¬ 
native  programs  under  consideration  would  have  to  be  "costed  out"  to 
reflect  their  complete  Incremental  resource  impact  for  the  long-term 
future--not  merely  the  "down  payment"  as  portrayed  by  next  fiscal  year's 
conventional  budget. 

All  of  this  sounds  conmonaenaical  enough.  It  is  basically  very 
similar  to  what  economic  planners  have  been  talking  about  for  years. 

In  ita  simplest  terms,  program  budgeting  is  primarily  the  identification 
and  systematic  examination  of  objectives  and  the  alternative  ways  of 
achieving  them.  The  main  focus  is  on  output- oriented  programs  or 
"packages  of  public  producr,"  not  the  input  orientation  of  the  conven¬ 
tional  budget  which  stresses  personnel,  equipment,  facilities,  trans¬ 
portation,  travel,  contractual  services,  and  the  like. 

Ir.  terms  of  the  current  jargon,  the  very  heart  of  program  budget¬ 
ing  la  contained  in  the  term  "systems  analysis."  While  systems  analy¬ 
sis  cannot  be  defined  with  precision,  the  following  would  probably  be 
accepted  as  a  reasonably  adequate  description  by  most  of  the  practi¬ 
tioners  today: 

*David  Novick  (ed.),  Program  Budgeting:  Program  Analysis  and  the 
Federal  Budget  (Cambridge,  Mass.:  Harvard  University  Press,  2d  ed., 
1967),  pp.  xvi-xix. 

itit 

Ibid.,  pp.  xxl-xxli. 


Systems  analysis  may  be  defined  as  Inquiry  to  assist  decision- 
makers  in  choosing  preferred  future  courses  of  action  by  (l) 
systematically  examining  and  re-examining  the  relevant  objec¬ 
tives  and  the  alternative  policies  or  strategies  for  achieving 
them;  and  (2)  comparing  quantitatively  where  possible  the  eco¬ 
nomic  costa,  effectiveness  (benefits),  and  risks  of  the  al¬ 
ternatives.  It  is  more  a  research  strategy  than  a  method  or 
technique,  and  in  its  present  state  of  development  it  is  more 
an  art  than  a  science.  In  sum,  systems  analysis  may  be  viewed 
as  an  approach  to,  or  way  of  looking  at,  complex  problems  of 
choice  under  conditions  of  uncertainty.* 


The  foregoing  provides  the  necessary  frame  of  reference  for  the 
discussion  of  the  main  subject  of  this  paper:  "Cost  Functions  and 
Budgets."  Perhaps  in  view  of  such  a  framework  a  more  descriptive 
title  would  be  "Cost  Considerations  in  Systems  Analysis."  In  any  event, 
the  latter  is  the  perspective  we  shall  stress. 


THE  KEY  ISSUES 

In  discussing  cost  analysis  as  a  part  of  systems  analysis,  it 
would  seem  that  we  could  segregate  the  main  Issues  into  two  categories: 

1.  The  conceptual  problems. 

2.  The  practical  problems  Involved  in  establishing  cost  analysis 
capabilities  in  the  Federal  Government  and  elsewhere. 

Both  of  these  are  important.  But  at  the  present  time  there  appears 
to  be  a  special  interest  and  sense  of  urgency  regarding  category  (2). 

We  shall  therefore  stress  (2)  In  this  paper.  Before  turning  to  such 
a  discussion,  however,  let  us  outline  briefly  some  of  the  characteris¬ 
tics  of  the  necessary  conceptual  framework  for  a  cost  analysis  capabil¬ 
ity  to  support  systems  analysis  studies. 

THE  CONCEPTUAL  FRAMEWORK 

In  large  measure  the  basic  concepts  underlying  systems  (program) 
cost  analysis  draw  very  heavily  on  concepts  taken  from  economic  theory 
and  analysis. 


For  a  further  discussion  of  systems  analysis,  see  E.  S.  Quade  and 
W.  I.  Boucher,  Systems  Analysis  and  Policy  Planning:  Applications  in 
Defense  (New  York:  American  Elsevier,  1968),  Chap.  I. 


A  representative,  but  far  from  complete,  listing  of  these  basic 
* 

concepts  is  as  follows: 

1.  An  explicit  relationship  between  inputs  and  outputs,  with  a 
strong  emphasis  on  output-oriented  identifications, 

2.  A  strong  emphasis  on  economic  (not  accounting)  costs.  Funda¬ 
mentally  this  r*«ans  the  concept  of  opportunity  cost. 

3.  As  a  further  elaboration  of  (2)  is  the  requirement  to  deal 
with  such  concepts  as: 

a.  Marginal  or  incremental  (and  hence  "sunk")  costa, 

b.  Fixed  and  variable  cnstR. 

c.  Recurring  and  non-recurring  costs. 

d.  Joiut  costs. 

4.  Explicit  treatment  of  uncertainty.  Simple  "expected  value" 
models  very  often  will  not  suffice. 

5.  An  awareness  of  scaling  considerations.  As  in  economic  theory 
many  problems  dealt  with  in  systems  cost  analyses  do  not  scale 
up  or  down  in  a  simple  fashion. 

6.  Explicit  treatment  of  problems  associated  with  time--e,g., 
the  problem  of  time  preference. 

7.  A  strong  emphasis  on  comparatl '  e  analyses  of  altematives-- 

e.g.,  fixed  cost  (budget)  com]  irisons  and/or  fixed  utility 
(effectiveness)  comparisons. 

8.  A  recognition  of  the  importance  of  sensitivity  analysis  con¬ 
tingency  analysis,  a  fortiori  argument,  and  the  Identification 
of  new  alternatives  as  ways  of  assisting  in  the  all  important 
search  for  dominances.** 

These  eight  points  represent  some  of  the  most  important  character¬ 
istics  of  the  conceptual  framework  for  cost  analysis  in  support  of  sys¬ 
tems  analysis  studies.  From  a  purely  conceptual  point  of  view,  there 
would  seem  to  be  little  room  for  argument  about  the  relevance  of  these 
concepts.  The  main  issues  arise  when  the  cost  analysts  try  to  Imple¬ 
ment  the  basic  ideas.  Let  us  consider  two  examples  briefly. 


The  ordering  in  no  way  reflects  relative  importance.  Also,  many 
of  tha  items  are  interrelated.  Finally,  in  soma  instances  it  is  not 
clear  that  the  particular  point  being  made  is  conceptual  or  methodolog- 
tcal--a  distinction  that  is  very  often  difficult  to  make. 


Perhaps  this  point  is  more  methodological 
Ua  Include  it  in  the  list  anyway,  because  of  its 
systems  analysis.  (Also,  it  is  strongly  related 
ment  of  uncertainty.) 


than  it  is  conceptual, 
central  Importance  in 
to  (4)--explicit  treat 
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Moot  analysts  agree  thet  In  principle  the  mat-ter  of  time  prefer¬ 
ence  Ah  mi  Id  he  treated  explicitly.  The  disagreement  arises  over  how 
this  should  be  done  fc>r  ex  ait  P 1  c .  with  respect  to  what  discount  rata 
fit"'- .  st  appropriate  fui  i  l  i_  .u,...  I'vtt  emit.  .,iouei- 

ous  seminars  and  conferences  have  been  held  on  the  subject  of  discuunl- 

★ 

ing,  and  the  issue  is  still  unresolved.  Many  analysts  ice  1  that  it 
cannot  be  completely  resolved  and  that  in  most  instance^  th.  matter  is 
less  consequential  than  many  of  the  other  cost' analysis  problems. 

In  fact,  the  analyst  can  do  a  great  deal  to  sharpen  the  intuition 

kk 

and  judgment  of  the  decisionmakers  without  resolving  the  rate  issue. 
toe  one  tiling,  tne  analyst  can  point  out  tr>  the  declsio—«tor8  that 
an  "undiscounted"  situation  usually  does  not  exist.  A  case  in  point 
occurs  in  the  Department  of  Defense  where  cost  streams  are  projected 
10  or  15  years  into  the  future  "without  equalizing  them  for  time  pref¬ 
erence."  Here,  the  analyst  can  make  the  time  preference  assumption 
explicit:  namely,  a  zero  percent  discount  rate  for  10  or  15  years  and 
a  very  high  rate  thereafter.  He  can  also  calculate  cases  built  on  a 
reasonable  range  of  time  preference  assumptions  and  show  the  impact  on 
final  results  (the  ranking  of  the  alternatives  being  considered).  Fi¬ 
nally,  he  can  compute  the  "breakeven  point"-  l.c.,  the  case  containing 
that  discount  rate  which  would  have  to  be  used  to  make  the  present  value 
of  two  alternatives  equal. 

As  a  second  illustration,  let  us  consider  the  concept  of  opportu¬ 
nity  cost.  Again,  there  would  seem  to  be  little  argument  about  the 
concept  itself.  Opportunity  costs  are  generally  recognized  as  being 
relevant  in  the  examination  and  evaluation  of  alternative  future  courses 
of  action.  They  are  certainly  much  more  relevant  than,  say,  accounting- 
type  costs  generated  for  fiduciary  financial  management  pr-foses. 


Recently  a  survey  was  made  of  23  Federal  agencies  to  obtain  in¬ 
formation  on  discounting  techniques  used  in  making  evaluations  of  fu¬ 
ture  government  programs.  Ona  of  the  results  is  that  the  rates  used 
vary  from  about  3  to  12  percent.  See  statement  by  Elmer  B.  Staats, 
Comptroller  General  of  the  United  States,  before  the  Subcotuni ttee  on 
Economy  in  the  Government,  Congressional  Record  -  Senate .  January  30, 
1968,  pp.  S632-S634. 

★  ★ 

To  sharpen  the  intuition  and  judgment  of  the  decisionmakers  is 
the  primary  role  of  systems  analysis.  Generally  speaking,  an  analysis 
cannot  make  the  decision.  (E.g.,  ace  A.  C.  Enthoven's  statement  con¬ 
tained  in  an  article  in  Business  Week.  November  13,  1965,  p.  189.) 
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Problems  arise,  however,  when  the  analysts  attempt  to  apply  the 
concept  of  opportunity  cost  in  systems  analysis  studies.  For  example, 
the  cost  analysts  often  generate  estimates  of  the  dollar  costs  of  vari¬ 
ous  program  or  system  alternatives  being  considered  in  long-range  plan- 

* 

ning  deliberations.  Such  estimates  may  be  expressed  in  terms  of  time- 
phased  expenditures  and/or  obligational  authority,  or  in  terms  of 
"static"  indexes  of  total  system  or  program  cost.  The  specific  issue 
is:  Do  these  dollar  cost  estimates  adequately  reflect  the  opportunity 
costs  (benefits  foregone)  of  the  alternatives  being  examined- -at  least 
for  the  purposes  of  the  types  of  comparisons  made  in  systems  analyses? 

Probably  not  in  all  cases.  However,  most  of  the  experts  seem  to 
think  that  for  the  purposes  of  comparing  distant  future  alternatives, 
dollar  costs  do  provide  a  reasonably  appropriate  index  of  opportunity 
costs  in  many  applications.  This  point  of  view  is  argued  vigorously, 
and  rather  convincingly,  in  the  context  of  the  Department  of  Defense 
by  Hitch  and  McKean: 

If  we  examine  the  problem  of  planning  future  programs  from 
the  standpoint  of  the  Defense  Department,  it  seems  fairly  ob¬ 
vious  that  money  costs  are  pertinent.  The  Department  faces 
a  budgetary  constraint.  For  the  most  part  it  does  not  face 
a  limitation  on  particular  weapons  or  supplies  but  can  buy 
more  of  them  by  paying  their  prices.  What  does  the  Depart¬ 
ment  give  up  in  order  to  implement  one  course  of  action? 

The  answer  is  money- -or,  to  go  one  step  further,  the  alter¬ 
native  weapons  or  supplies  that  could  otherwise  be  purchased. 

The  Department  could  substitute  one  item  for  another  by  pay¬ 
ing  the  price  of  the  one  instead  of  paying  the  price  of  the 
other.  Dollar  costs  do  reflect  what  must  be  given  up  in  or¬ 
der  to  adopt  a  particular  policy.  They  reflect  real  sacri¬ 
fices  by  the  Department  beca>  ce  the  prices  of  different  items 
show  the  rates  at  which  they  can  be  substituted  for  each 
other. 

This  is  not  to  say  that  money  costs  perfectly  represent  re¬ 
sources  sacrificed  by  the  Defense  Department.  The  prices 
of  goods  to  be  bought  in  the  future  are  uncertain.  One  course 
of  action  may  itself  drive  up  the  price  of  particular  weapons 


Other  measures  of  cost  are  also  calculated--e.g. ,  manpower. 

That  is,  the  sum  of  development  (if  any)  and  investment  costs 
plus  a  number  of  years  operating  cost. 
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or  materials,  and  it  is  not  possible  to  predict  these  ef¬ 
fects  with  complete  accuracy.  The  characteristics  and  cost 
of  some  items  will  change  as  technology  advances.  The  quan¬ 
tity  of  some  exceptional  items  may  literally  be  fixed,  or 
nearly  fixed,  even  if  we  are  looking  several  years  ahead. 

Nonetheless,  imperfect  as  it  is,  the  money  cost  of  a  future 
program  usually  shows  the  sacrifice  that  would  be  required 
of  the  Department  better  than  other  measures  of  cost.  While 
dollars  do  not  precisely  measure  the  real  sacrifices,  costs 
in  terms  of  metals  and  manpower  would  be  grossly  misleading. 

Saying  that  airplants  cost  so  much  aluminum  and  ships  so 
much  steel  plate  does  not  tell  us  how  one  may  be  exchanged 
for  the  other.  Saying  that  each  costs  so  many  dollars  ad¬ 
heres  more  closely  to  the  facts,  namely,  that  the  services 
can,  in  making  future  plans,  trade  one  for  the  other.* 

Our  concern  in  this  paper  is  not  so  much  whether  Hitch  and  McKean 
are  correct.  Rather  the  point  is  that  the  issue  under  consideration 
pertains  more  to  the  matter  of  imp lementation  of  a  concept  rather  than 
to  the  relevance  of  the  concept  itself.  It  is  generally  recognized 
that  economic  cost  is  one  of  the  relevant  considerations  to  be  taken 
into  account  in  systems  analysis  studies.  The  question  is  how  to  do 
it  in  practice. 

This  leads  to  our  next  subject:  cost  analysis  in  support  of  sys¬ 
tems  analysis  in  practice.  How  is  it  done?  What  are  some  of  the  problems? 


COST  ANALYSIS  IN  PRACTICE 


In  general  terms  the  central  problem  facing  cost  analysts  is  to 
develop  methods  and  techniques  which  will  permit  assessment  of  the 


Charles  J.  Hitch  and  Roland  N.  McKean,  The  Economics  of  Defense 
in  the  Nuclear  Axe  (Cambridge,  Mass.:  Harvard  University  Press,  1960), 
p.  26. 

The  above  quotation  is  concerned  with  dollars  as  a  proximate  meas¬ 
ure  of  economic  cost  from  the  viewpoint  of  the  Department  of  Defense. 
What  about  from  the  standpoint  of  the  nation?  Hitch  and  McKean  consider 
this  question,  and  argue  the  c«se  by  means  of  an  illustrative  example. 
(See  ibid.,  pp.  27-28.)  Their  general  conclusion  is: 

As  a  consequence,  money  costs  of  future  defense  activities  ap¬ 
proximate  the  real  alternatives  that  are  foregone--the  real 
sacrifices  that  are  entailed — when  one  activity  or  weapon  sys¬ 
tem  is  selected.  This  will  be  true  for  those  problems  in 
which  a  general  monetary  constraint  is  proper,  that  is,  for 
problems  pertaining  to  dates  sufficiently  in' the  future  to 
permit  the  production  and  procurement  of  varying  quantities 
of  weapons  and  materiel.  (Ibid. .  p.  28.) 


resource  Impact  of  proposed  alternative  output-oriented  programy  and/or 
alternative  combinations  (mixes)  of  future  programs. 

The  basic  characteristic*  of  such  a  cost  analysis  capability  bcse 
directly  fro®  the  conceptual  framework  discussed  In  the  preceding  sec¬ 
tion,  A  few  ot  the  more  important  of  these  characteristics  are  ay 
follows ; 


1.  While  most  of  the  basic  estimating  work  must  be  done  on  the 
input  side  In  tenaa  of  manpower,  equipment,  facilities,  sup¬ 
plies,  etc.,  the  results  of  a  cost  analysis  oust  be  packaged 
in  the  fora  of  output-oriented  entities  which  are  of  prime 
concern  to  the  long-range  planning  decisionmakers . 

2.  Cost  analysis  procedures  (models)  oust  be  "open-ended"  with 
respect  to  key  performance  and  other  variables  which  charac¬ 
ter!:;?  the  class  of  output-oriented  entities  under  considera¬ 
tion.  This  facilitates  doing  parametric- type  analyses  which 
are  of  prims  importance  in  extending  the  range  of  alternatives 
that  can  be  examined,  in  making  a  fortiori  arguments,  in  stak¬ 
ing  sensitivity  tests,  in  exploring  scaling  factors,  and  the 
like. 

3.  Related  to  (2)  is  the  requirement  to  deal  explicitly  with  the 
problem  of  uncertainty.  Parametric  coat  models  help  by  facil¬ 
itating  the  computation  of  a  range  of  coat  estimates  (rather 
than  "point"  estimates  alone),*  and  by  permitting  determina¬ 
tion  of  the  sensitivity  of  total  system  or  prsgram  cost  to 
variations  in  the  values  of  key  parameters  about  which  we  are 
uncertain. 

4.  Both  (2)  and  (3)  imply  the  need  for  cost  analysis  models  which 
are  in  part  automated.  If  a  large  number  of  cases  arc  to  be 
computed  within  a  reasonable  amount  of  time  and  effort,  man¬ 
ual  calculation  alone  is  usually  out  of  the  question.** 

5.  A  strong  emphasis  must  be  placed  on  developing  cost  analysis 
procedures  which  will  permit  ssseBsment  of  Incremental  costs-- 
i.e.,  the  additional  costs  implied  by  the  proposed  future 
course  of  action  under  consideration.  The  costs  of  past  ac¬ 
tions  ("sunk"  costs)  and  the  costs  of  firmly  committed  ("locked 
in")  future  actions  must  be  excluded. 


*Foe  example .  "high,"  "medium"  and  "low"  cases. 

**Xn  seme  Instances  a  tremendous  number  of  individual  calculations 
ars  required  for  a  single  case.  For  example,  the  total  force  structure 
coat  sKidal  developed  by  The  RAMD  Corporation  to  assess  the  resource  im¬ 
pact  of  s  projected  total  Air  Pores  plan  over  e  10- year  period  makes 
about  500,000  computations  for  the  typical  single  csss. 
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6.  Finally--and  perhaps  most  important  of  all — a  substantial 
amount  of  time  and  effort  must  be  devoted  to  the  continuous 
development  and  maintenance  of  an  appropriate  data  base:  i.e., 
information  on  past,  current  and  near  future  programs  to  serve 
as  a  basis  for  the  derivation  of  estimating  relationships  to 
be  used  in  projecting  to  the  distant  future. 

We  have  stressed  the  prime  importance  of  "output-oriented  enti¬ 
ties"  or  "piogram  packages."  What  does  this  mean  in  areas  of  specific 
application?  In  the  case  of  the  Department  of  Defense  these  entities 
are  by  now  rather  widely  recognized  as  being  weapon  and  support  systems 
and  force  mixes  of  such  systems.  What  about  the  non-military  realm? 

The  following  are  a  few  examples. 

In  the  transportation  area  output-oriented  entities  may  be  vari¬ 
ous  future  modes  of  transportation--rail  systems  (surface  and  sub¬ 
surface),  "automated"  freeway  systems,  airlift  systems,  etc.,  and  mixes 
of  these  modes.  In  the  mental  health  area  the  planners  may  be  inter¬ 
ested  in  such  things  as  alternative  systems  for  dispensing  mental  health 
services  (e.g.,  community  mental  health  centers),  alternative  programs 
for  narcotic  and  drug  abuse,  etc.  In  the  National  Aeronautics  and 
Space  Administration  the  prime  concern  is  with  alternative  ways  of  at¬ 
taining  certain  goals  in  spac.e--e.g.,  alternative  space  systems  for 
performing  future  missions  in  the  lunar,  earth-orbital,  and  planetary 
areas . 

In  any  event,  the  cost  analyst  must  be  able  to  conduct  his  stud¬ 
ies  in  terms  of  the  types  of  identifications  or  "planning  units"  that 
are  of  primary  interest  Vo  f.e  long-range  planners.  This  is  just  as 
true  in  non-national  security  problem  areas  as  it  is  in  defense,  and 
the  basic  problems  are  very  similar. 

Let  us  now  consider  a  few  hypothetical  examples  to  illustrate  some 
of  the  types  of  output  from  the  cost  analysis  process  which  are  useful 
in  systems  analysis.  In  these  illustrations  the  "output-oriented  enti¬ 
tles"  are  assumed  to  be  alternative  system  or  program  package  proposals 
being  considered  in  the  long-range  planning  process. 

A  very  useful  output  of  the  cost  analysis  process  (an  input  to 
systems  analysis)  is  a  cost  function  relating  projected  total  system 
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(program)  cost  to  the  size  (cumulative  number  of  units)  of  a  proposed 
future  course  of  action.  An  example  is  shown  in  Fig.  I,  Here,  total 
system  cost  is  increasing  at  a  decreasing  rate,  and  hence  marginal  erst 
(the  cost  of  an  additional  unit)  is  a  decreasing  function  of  the  cumu¬ 
lative  number  of  units.  This  is  portrayed  in  Fig.  2. 

Cost  functions  expressing  to.ul  system  cost  as  a  function  of  cumu¬ 
lative  number  of  units  are  particularly  useful  in  a  fixed  budget  com¬ 
parative  framework  of  analysis.  Here,  the  systems  analyst  often  wants 
to  know:  "Hov  many  units  of  the  various  alternatives  under  consJdera- 

-a  i 

tion  can  I  ge^  out  of  certain  stipulated  future  budget  levels'?" 

An  example  is  presented  in  Fig.  3  for  alternatives  A  and  B.  If 
the  specified  cost  level  to  be  used  in  the  comparative  analysis  is 
$8  tilllon,  11.5  units  of  alternative  A  or  7  units  of  alternative  B  are 
obtainable.  This  is  a  key  output  of  the  cost  analysis,  which  then  be¬ 
comes  a  major  input  to  the  ef fectivenesr  (utility)  analysis. 

Notice  that  in  this  illustration  the  results  do  not  scale  linearly 
with  respect  to  changes  in  the  stipulated  cost  level.  For  example,  if 
Is  increased  by  50  percent  to  L 2  ■  $12  billion,  the  outcome  is  22 
units  of  A  or  12  units  of  B.  The  increase  in  the  number  of  units  is 
greater  than  the  increase  in  over  L^: 

Lj/Lj  -  12/8  -  150% 

A2/A1  -  22/11.5  * **  1917. 

B2/B1  “  12/7  -  1717. 

*Kere,  total  system  (program)  cost  is  defined  as  development  (if 
any)  plus  initial  Investment  plus  a  fixed  number  of  future  years  opera¬ 
ting  cost.  Oftentimes  the  number  of  years  operation  is  treated  para¬ 
metrically  to  see  whether  the  assumption  about  this  factor  impacts  sig¬ 
nificantly  upon  final  results  (the  ranking  of  the  alternatlvec  being 
considered) . 

**Thie  is  one  example  of  way  cost  functions  relating  cost  to  the 
scale  of  proposed  future  programs  are  useful  in  systems  analysis  work. 

Many  of  the  cost  functions  emphasized  in  conventional  economic 
theory  relate  cost  to  rate  of  output.  Rate-of-outpue  cost  functions 
are  also  uaeful  In  certain  types  of  problems  In  systems  analysis.  For 
example,  in  studying  alternative  configurations  of  proposed  community 
mental  health  centers,  we  might  want  to  examine  hov  unit  costs  change 
as  dally  out-patient  capacity  rate  la  varied  over  a  relevant  range, 

In  the  case  of  military  aircraft  systems  the  analyst  often  examines  how 
system  cost  (for  a  fixed  force  size)  changes  as  the  activity  rate  (e.g., 
flying  hours  per  aircraft  per  month)  is  varied  over  a  certain  range. 
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Fig.  3 — Total  system  cost  vs.  program  size 
for  alternatives  A  and  B 


-14- 


In  a  simple  way  this  demonstrates  that  in  the  context  of  a  fixed 
budget  framework  of  analysis,  scaling  considerations  suggest  the  de¬ 
sirability  of  conducting  the  comparisons  for  more  than  one  cost  level. 
For  example,  three  cases  might  be  examined:  high,  medium,  and  low. 

To  illustrate  a  somewhat  different  point,  let  us  now  consider  a 
fixed  effectiveness  framework  of  analysis  for  comparing  alternative 
proposed  future  courses  of  action.  Here,  the  analysis  attempts  to  de¬ 
termine  that  alternative  (or  feasible  combination  of  alternatives) 
which  is  likely  to  achieve  some  specified  level  of  effectiveness  at 
the  lowest  economic  cost.  The  cost  analysis  in  effect  produces  the 
final  results  after  the  effectiveness  analysis  has  determined  how  much 
of  each  alternative  is  required  to  attain  the  stipulated  level  of 
effectiveness . 

As  one  simple  illustration  of  this  approach,  suppose  that  two  al¬ 
ternatives  C  and  D  are  under  consideration,  and  that  the  results  of  the 
effectiveness  analysis  indicate  the  following  ranges  of  quantities  (num¬ 
ber  of  units)  of  C  and  D  required  to  attain  some  specified  level  of  ef¬ 
fectiveness  Eq  i 

_C  _D 

Low  20  4 

Expected  value  22  6 

High  24  12 

Notice  that  in  this  case  the  range  for  D  is  considerably  greater  than 
for  C  because  of  uncertainty. 

Suppose  now  that  the  estimated  total  system  costs  as  a  function 

of  cumulative  number  of  units  for  C  and  D  are  as  shown  in  Fig.  4, 

Taking  the  expected  value  outputs  from  the  effectiveness  analysis,  we 
see  from  Fig.  4  that  D  is  the  least  cost  alternative  for  attaining  ef¬ 
fectiveness  level  Eq:  $7.5  billion  for  D  vs.  $15.3  billion  for  C,  or 

a  factor-of-two  difference  in  favor  of  D.  If  the  uncertainties  in  the 

effectiveness  analysis  arc  taken  into  account,  alternative  D  still 
holds  up  well,  even  in  the  situation  where  the  worst  case  (highest  cost) 
for  D  and  the  best  case  (lowest  cost)  for  C  are  paired  up.  Thus,  at 
least  with  respect  to  the  uncertainties  taken  into  account  in  the  prob¬ 
lem,  alternative  D  appears  to  be  s  dominant  solutlon--something  which 
the  systems  analyst  is  always  seeking,  but  rarely  finds. 
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Fig.  4 — Total  system  cost  vs.  program  size 
for  alternatives  C  and  D 

(Fixed  effectiveness  »  K  ) 
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So  far,  our  examples  have  been  essentially  "static":  i.e.,  pro¬ 
jected  costs  have  not  been  treated  explicitly  as  a  function  of  tine. 

Xu  assy  decision  contexts,  however,  the  alternatives  have  to  be  exee>- 

inad  In  terms  of  time-phased  cost  streams  projected  a  number  of  years 
into  the  future. 

As  an  example,  suppose  there  are  tvo  new  proposed  alternatives, 
programs  E  and  F,  which  are  estimated  to  be  capable  of  accomplishing 
the  same  objective  in  the  future  with  essentially  the  same  degree  of 
effectiveness  for  the  time  period  of  interest.  (A  "fixed-effectiveness 
over  time"  framework  of  analysis.)  Suppose  further  that  the  time- 
phased  total  program  coats  over  a  15-year  period  in  the  future  are  as 
portrayed  in  Fig.  5.  Here,  the  time  preference  assumption  is  a  zero 
discount  rata  for  the  first  15  years,  and  a  very  high  rate  (over  100 
percent)  thereafter.  Notice  that  in  each  case  when  the  yearly  costs 
era  summed  over  the  15-year  period,  the  totals  are  the  same  ($9  billion 
each  for  E  and  F). 

On  the  basis  of  the  data  presented  so  far,  we  have  an  equal- 
effectiveness,  equal-coat  situation;  so  presumably  the  decisionmakers 
would  be  indifferent  regarding  the  choice  of  E  or  F--at  least  on  the 
basis  of  the  quantitative  information  available  at  thli  point. 

Notice,  however,  that  the  time  impacts  of  the  coats  for  E  and  F 
are  considerably  different.  The  basic  reason  for  the  difference  is 
that  alternative  E  requires  higher  cosc  outlays  (relative  to  F)  early 
in  the  period  because  of  greater  development  and  investment  costs. 

These  outlays  pay  off  in  terms  of  an  efficient  operational  program  hav¬ 
ing  relatively  low  operating  costs  later  In  the  period.  Alternative  F, 
on  the  other  hand,  has  lover  development  and  investment  costs  than  E. 
This,  however,  implies  a  lass  efficient  operational  program  than  E, 
with  the  result  that  larger  operating  coats  are  required  to  accomplish 
the  specified  task  with  the  aasw  degree  of  effectiveness  aa  E.  There¬ 
fore,  the  costa  for  F  during  the  latter  years  of  the  15-year  period  are 
about  2  times  those  of  E. 

In  view  of  these  differences  in  the  time  iapact  of  the  coats  of  E 
and  F,  the  question  arises  as  to  whether  the  planners  would  still  be 
indifferent  regarding  the  choice  of  E  or  F  if  the  clam  preference 


Program  Cost  (Billions  of  1967  Dollar 


18- 


assureptlons  are  varied.  Fuppose  the  base  case  (Fig.  5)  Is  modified  to 
reflect  the  following  tine  preference  spec  If  icatiuna ;  a  discount  rule 
of  6  percent  for  15  years,  and  a  very  high  rate  (over  1 0(77- )  thereafter. 
The  results  are  as  follows: 


Alternative  £ 
Alternative  F 
Difference 


Present  Value  Coat  in  Billions 
Base  Case  First  Modification 


(0;  1QQ+7. ) 
$9 
_9 
0 


(6;  100+7.  )a 
$5.8 
5.3 
0.5 


*The  time-phased  cost  profiles  for  this  case  are  presented 
in  Fig.  6. 


Thus,  the  first  modification  (67.  for  the  first  15  years)  results 
In  a  rather  sharp  reduction  In  the  present  value  of  the  15-year  costs 
for  both  E  and  F.  However,  the  difference  between  them  can  hardly  be 
regarded  es  significant  In  view  of  the  many  uncertainties  involved  in 
the  total  analysis.  The  decisionmakers  are  likely  to  continue  to  be 
Indifferent  regarding  the  choice  of  E  or  F  on  the  basis  of  the  present 
values  of  the  two  cost  streaau. 

Would  this  still  be  the  case  for  a  discount  rate  considerably 
higher  than  6  percent?  Let  us  try  a  10  percent  rate  for  15  years,  and 
a  very  high  rate  (over  100%)  thereafter.  The  results  are: 


Present 

Value  Cost  in 

Billions 

Base  Case 

(0;  100+7, ) 

First  Mod, 
(6;100-F7.) 

Second  Mod, 

Alternative  E  -  • 

•  -  -  -  $9 

$5.8 

$4.5 

Alternative  F  -  ■ 

■  -  -  -  _j> 

5.3 

3.9 

Difference  -  ■ 

•  -  -  -  0 

0.5 

0.6 

* m 

The  second  modification  results 

in 

a  further  reduction  in  the 

present  values  of  the 

15-year  coats 

for 

both  E  and  F. 

Here  again  it 

la  very  doubtful  that 

the  difference 

in 

present  values 

between  E  and 

is  significant.  The  uncertainties  in  the  basic  problem  are  such  that 
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■  15  percent  difference  In  cost  Is  no  doubt  well  within  the  "nulse  Level" 

of  the  analysis .  Also  It  should  be  pointed  nut  that  in  most  contexts  a 

* 

10  percent  discount  rate  for  time  preference  Is  fairly  high,  which  makes 
makes  the  second  modification  a  rather  extreme  case. 

In  sum,  In  this  particular  example,  the  conclusion  would  ueem  to 
be  that  the  decision  regarding  the  choice  of  alternative  E  or  T  Is 
likely  to  b«  Independent  of  the  assumptions  made  with  respect  to  the 

kir 

treatment  of  time  preference. 

As  a  final  example,  let  us  consider  the  examination  of  variations 
In  total  system  (program)  cost  as  the  characteristics  of  the  system 
(program)  are  varied,  assuming  a  fixed  number  of  years  of  operation. 

This  represents  an  Important  technique  of  analysis  in  systems  analysis. 

For  an  Illustration,  we  shall  use  the  context  of  the  national  se¬ 
curity  area  and  consider  the  case  of  a  proposed  future  aircraft  system 
where  the  mission  requires  that  a  fleet  of  aircraft  be  continuously 
airborne  on  a  aeries  of  stations  which  cover  a  large  geographical  area. 

A  Navy  antisubmarine  warfare  (ASW)  mission  in  the  future  is  a  possible 
example . 

Continuously  airborne  alert  aircraft  systems  typically  involve  a 
host  of  significant  variables:  endurance  hours  of  the  aircraft  to  be 
employed  in  the  system,  extent  of  the  area  coverage,  nature  of  the 

♦ 

Recall  that  in  this  exercise  we  have  been  dlacounting  for  time 
preference  only--not  for  time  preference  plus  a  supplemental  rate  for 
risk.  When  analysts  apply  rates  like  10  to  15  percent,  they  usually 
have  in  mind  a  combined  rate  to  allow  for  time  preference  and  risk  or 
uncertainty. 

krk 

Examples  of  other  cases  arc  the  following: 


Present  Value  in  Billions 


Case 

Alt.  E 

Alt.  F 

Difference 

61 

for  25  yr  -  -  -  - 

■  -  -  - 

$6.9 

$7.4 

$0.5 

107, 

for  25  yr  -  -  -  - 

■  -  -  - 

5.0 

4.9 

0.1 

157, 

for  25  yr  -  -  -  - 

■  -  -  - 

3.5 

3.1 

0.4 

51 

for  first  10  yr. 

iat 
20 1 

for  next  5  yr  I 
for  next  5  yr  | 

-  -  - 

6.6 

6.5 

0.1 

501 

for  next  5  yr  ' 

I0fc 
5  OX 

for  first  15  yr, 
for  yr  16-25  1 

... 

4.6 

4.2 

0.4 
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puyload  requirements,  aircraft  maintenance  policy  (out,  two,  or  three 
shifts),  and  the  like.  Intra-system  cost  analysis  must  usually  explore 
the  consequences  nf  variations  in  these  variables. 

Figure  /  shows  an  example  for  a  future  ASW  system  to  patrol  and 
destroy  ballistic  missile-carrying  enemy  submarines,  where  aircraft  en¬ 
durance  hours  and  area  coverage  (nautical  miles  out  to  sea  from  U.S. 
coastlines)  are  varied.  Here  total  system  cost  is  defined  to  be  re¬ 
search  and  development  +  investment  +  five  years  of  operation,  Not  ire 
that  as  the  area  coverage  la  extended,  the  requirement  for  longer  en¬ 
durance  becomes  increasingly  more  severe. 

Figure  8  contains  another  ASW  system  cost  example.  Here  total  sys¬ 
tem  cost  (defined  as  in  Fig.  7)  for  each  pound  of  payload  (e lectronica , 
ASW  missiles,  etc.)  on  station  Is  expressed  as  a  function  of  the  pounds 
of  payload  carried  per  aircraft.*  Curves  are  shown  for  three  types  of 
aircraft  that  might  be  candidates  for  use  in  the  proposed  ASW  system. 

Notice  that  the  use  of  conventional  jets  in  this  mission  applica¬ 
tion  results  in  a  considerably  higher  minimum  cost  point  than  for  long- 
endurance  aircraft,  and  that  system  cost  per  pound  of  payload  on  station 
is  very  sensitive  to  individual  aircraft  payload  weight.  Note  also 
that  as  we  move  to  the  large,  long-cndurance  aircraft,  the  coats  become 
much  less  sensitive  to  a  particular  loading  or  payload  weight.  This 
might  suggest  that  if  the  size  of  the  payload  to  perform  the  future 
mission  is  clouded  by  uncertainties,  then  flexibility  may  be  achieved 
by  going  to  the  large,  long-endurance  aircraft. 

A  MAJOR  DIFFICULTY :  THE  DATA  PROBLEM 

What  are  the  main  difficulties  involved  in  doing  cost  analyses  of 
the  type  described  in  the  previous  section?  There  are  many-some  bureau¬ 
cratic,  some  substantive.  Because  of  space  limitations,  all  of  these 
difficulties  cannot  be  outlined  and  discussed  here.  We  shall  therefore 

* 

Area  coverage  is  fixed  at  1000  nautical  miles. 


Fig.  7 — ASW  system  cost  vs.  aircraft 
endurance  for  several  area 
coverages 


Synthetic  number  scale 


Total  System  Cost  per  Pound 


Fig.  8 — System  cost  pet  pound  of  payload  on  station 
vs.  aircraft  payload  weight 

(Area  coverage  =  1000  n.mi.) 


Synthetic  number  scale 
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take  the  moat  severe--the  data  problem--and  treat  it  at  some  length. 

In  moat  caaea  the  ability  to  engage  in  cost  analysis  as  an  inte¬ 
gral  part  of  systems  analysis  studies  requires  the  development  and  use 
cf  coat  models.  A  cost  model  is  a  device  for  generating  estimates  of 
the  resource  impact  of  future  output-oriented  program  packages  in  terms 
of  the  Inputs  that  would  be  required  to  develop,  install,  and  operate 
these  proposed  programs  over  a  period  of  years.  The  input  structure 
typically  involves  various  categories  of  facilities,  equipment,  per¬ 
sonnel,  supplies,  etc.,  or  combinations  of  these  items  (maintenance, 
for  example). 

For  each  category  in  the  input  structure  we  must  have  an  estima¬ 
ting  relationship  (or  series  of  relationships)  expressing  cost  as  a 

function  of  an  appropriate  set  of  cost-generating  or  explanatory  -ari- 
** 

ables.  These  estimating  relationships  form  the  very  heart  of  a  cost 
analysis  capability. 

Estimating  relationships  have  to  be  derived  cm  the  basis  of  some¬ 
thing.  Sometimes  that  something  has  to  be  "experience  and  judgment" 
(preferably  of  an  expert).  Generally  speaking,  however,  we  would  pre¬ 
fer  that  they  be  developed  from  statistical  analyses  of  past,  current, 
and  near  future  data  and  information.  At  this  point  we  run  headlong 
into  "the  data  problem."  And  it  ia  a  problem  of  fundamental  Importance, 
because  a  substantive  cost  analysis  capability  cannot  exist  without  an 
appropriate  Information  and  data  bank. 

* 

Much  of  the  discussion  to  follow  draws  rather  heavily  on  the  na¬ 
tional  security  area.  Ibis  is  because  the  author  has  had  more  experi¬ 
ence  in  the  Department  of  Defense  than  in  other  governmental  agencies. 
The  basic  data  problems,  however,  are  very  similar  in  all  areaB,  and 
the  methodological  and  procedural  points  to  be  made  in  this  section  are 
applicable  to  a  wide  variety  of  contexts. 

“frit 

The  cost  of  a  certain  type  of  equipment  for  the  future  may  be 
estimated  as  a  function  of  its  performance  and/or  physical  character¬ 
istics  and  estimated  production  quantity.  The  cost  of  equipment  main¬ 
tenance  may  be  estimated  as  a  function  of  equipment  characteristics  and 
projected  activity  rate. 
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Whv  Is  There  a  Data  Problem? 

The  reader  may  well  wonder  why  the  data  problem  is  eo  severe.  He* 
not  the  Government  been  developing  information  systems  and  collecting 
a  huge  volume  of  data  in  numerous  areas  for  many  years?  Has  not  indus¬ 
try  and  other  institutions  been  doing  the  same  thing?  How  could  there 
be  a  "data  problem"? 

These  are  legitimate  questions.  The  answers  are  numerous  and  var¬ 
ied.  Here,  we  shall  try  to  select  a  few  of  the  more  important  ones, 

with  a  view  to  giving  the  reader  a  reasonable  degree  of  understanding 

★ 

of  why  a  data  problem  exists. 

Information  in  the  wronR  format.  Information  systems  in  the  Gov¬ 
ernment  and  elsewhere  have  indeed  generated  a  tremendous  amount  of  data. 
In  many  instances,  however,  these  data  are  not  in  an  appropriate  format 
to  be  very  useful  in  a  progiam  cost  analysis  activity  serving  the  long- 
range  planning  process. 

The  main  reason  for  this  is  that  these  information  systems  were 
established  primarily  to  serve  the  needs  of  managers  of  functional  areas 
of  operational  activity  (maintenance,  supply,  etc.),  of  managers  respon¬ 
sible  for  fiscal  integrity  or  fiduciary  accounting  requirements  ("keep¬ 
ing  hands  out  of  the  till"),  of  managers  concerned  with  critical  re¬ 
source  items  across  the  board  (e.g.,  personnel),  of  budgeteers  concerned 
with  the  conventional  budget,  and  the  like.  In  short,  the  orientation 
of  a  large  number  of  past  and  existing  Information  systems  is  toward 
the  input  aide  per  se ,  with  little  or  no  provision  for  making  meaning¬ 
ful  translations  reflecting  impacts  on  output-oriented  program  packages. 


The  points  discussed  below  relate  for  the  most  part  to  technical 
aspects  of  the  problem.  Other  factors  can  be  important  also.  For  ex¬ 
ample,  formal  information  and  data  systems  are  sometimes  established 
without  sufficient  understanding  of  the  relevant  organizational  and  In¬ 
stitutional  considerations  pertaining  to  the  agency  in  question. 

★* 

Oftentimes  the  suggestion  Is  made  that  if  the  analyst  will  probe 
the  data  base  at  successively  greater  levels  of  detail,  he  will  eventu¬ 
ally  find  the  kinds  of  identifications  he  needs.  Sometimes  this  is 
true.  On  the  other  hand,  one  Is  likely  to  find  that  if  an  information 
system  la  structured  to  deal  in  terms  of,  say,  "object  classes,"  then 
going  into  more  detail  will  simply  yield  greater  amounts  of  information 
in  the  same  terms  (object  classes). 
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The  "patching  up"  or  Integration  problem.  Particularly  when  the 
objective  is  to  derive  estimating  relationships,  the  analyst  must  not 
only  collect  historic. 1  cost  data  in  the  right  format.  He  must  also 
obtain  information  on  quantities,  physical  and  performance  character¬ 
istics,  activity  rates,  and  other  types  of  cost-generating  variables. 

The  latter  must  be  matched  specifically  to  the  cost  data  points. 

Sometimes  this  is  difficult  because  the  information  on  the  cost- 
generating  variables  must  be  extracted  from  different  sets  of  records 
than  those  containing  the  cost  data.  And  differing  sets  of  records  can 
often  have  dissimilar  bases  for  reporting--for  example,  with  respect 
to  lot  size,  time  period  covered,  and  the  like. 

Differences  in  definitions  of  categories.  A  different  kind  of 
"matching  up"  problem  frequently  occurs.  This  concerns  the  lack  of  a 
one-to-one  correspondence  between  the  definition  of  the  content  of  cate¬ 
gories  in  the  input  structures  set  up  for  program  cost  analysis  purposes, 
and  the  definition  of  analogous  categories  in  the  existing  data  and 
information  collection  systems. 

It  la  not  possible  to  aet  up  the  preferred  input  structure  which 
will  meet  the  requirements  of  cost  analyses  in  support  of  long-range 
planning  and  at  the  same  time  be  in  complete  harmony  with  existing  data 
and  information  ayatema  at  any  point  in  time.  Differences  in  defini¬ 
tion  of  certain  categories  in  the  input  structure  and  their  counter¬ 
parts  in  the  existing  data  base  are  therefore  bound  to  be  present.  This 
creates  a  data  problem  for  the  cost  analyst  when  he  is  collecting  in¬ 
formation  to  serve  as  the  basis  for  deriving  estimating  relationships 
for  various  categories  and  sub-categories  in  his  input  structure.  He 
will  often  have  to  make  adjustments  to  the  raw  data  to  correct  for  these 
definitional  differences. 

The  influence  of  temporal  factors.  Historical  data  are,  of  course, 
generated  over  time.  This  means  that  numerous  dynamic  factors  will 
have  influences  on  the  information  being  collected  in  a  certain  area. 
Flrat  of  all,  the  Information  collection  systems  themselves  have  a 
habit  of  changing  over  tlme--for  example,  the  appropriate  definition 
of  the  content  of  various  categories  b*:ing  used  to  accumulate  the 
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hlstorleal  data  may  change  as  the  system  evolves.  Also,  in  the  case 
of  financial  data,  price  level  changes  will  occur  and  be  reflected  in 
the  information  being  collected  over  time. 

In  addition  to  these  types  of  temporal  considerations  is  the  im¬ 
portant  fact  chat  many  Governmental  agencies  deal  with  a  rapidly  chang¬ 
ing  technology,  both  with  respect  to  hardware  and  with  respect  to  or¬ 
ganizational  and  operational  concepts.  Almost  by  definition  this  means 
that  even  with  a  near  perfect  information  collection  system,  only  a 
relatively  small  Bample  of  data  can  be  generated  for  a  given  era  or 
class  of  technology.  In  the  major  equipment  area,  for  example,  the 
analyst  ia  lucky  if  he  can  have  available  15  or  20  good  data  points  for 
a  certain  class  of  hardware.  He  is  more  likely  to  have  less  than  half 
that  number. 

By  the  nature  of  things,  therefore,  the  analyst  is  all  too  often 
in  the  world  of  very  small  samples.  As  all  good  statisticians  know, 
this  poses  real  problems  in  our  attempts  to  develop  meaningful  struc¬ 
tural  relationships  which  will  permit  us  to  project  forward  to  distant 
future  programs  and  capabilities. 

So  much  for  our  listing  of  problem  areas  concerning  the  data  base. 
We  repeat  that  the  four  points  outlined  above  do  not  represent  a  com¬ 
plete  enumeration;  they  should,  however,  convince  the  reader  that  there 
is  such  a  thing  as  a  "data  problem."  The  question  now  is:  What  do  we 
do  about  it? 

DEALING  WITH  THE  DATA  PROBLEM 

At  first  thought,  one  might  be  tempted  to  say:  "If  there  is  a 
data  problem,  let's  solve  it  once-for-all  by  establishing  the  informa¬ 
tion  collection  system  to  meet  all  our  needs."  People  have  often  made 
statements  like  this.  Is  such  a  thing  feasible? 

We  think  not,  for  several  reasons.  Some  of  the  more  important  of 
these  are  the  following: 

1.  Cost  analysis  problems  in  support  of  systems  analyses  typically 
vary  considerably  from  one  study  to  another.  The  requirements  for  es¬ 
timating  relatlonships--and  hence  data  and  information  requlramants-- 
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are  not  constant  over  time,  or  even  for  a  given  small  Interval  of  time. 
In  short,  the  cost  analyst  who  is  working  in  support  of  the  long-range 
planning  process  could  not  specify  his  data  and  information  needs  "once- 
for-all."  It  would  be  difficult,  if  not  impossible,  then,  to  establish 
tha  comprehensive  information  system, 

2.  Even  if  something  approaching  (1)  could  be  done,  we  still  have 
to  worry  about  economics.  Large  information  syatcms--cspeeifll ly  those 
designed  for  complete  enumerations --are  very  expensive.  This  poses  a 
system;  analysis  problem  in  itself.  Would  the  (large)  incremental  cost 
of  a  new  complete  enumeration  information  system  be  Justified  in  terms 
of  the  benefits  to  be  dcrived--perticular ly  in  the  long-range  planning 
context  where  high  precision  in  en  absolute  sense  is  usually  not  a  prime 
requirement?  The  answer  is  probably  "no." 

3.  In  addition  to  points  (1)  and  (2)  is  tne  problem  of  small  sam¬ 
ples  erislng  from  the  fact  that  many  Governmental  agencies  have  to  deal 
with  a  rapidly  changing  technology.  As  indicated  previously,  this 
skeans  that  in  many  instances  only  a  relatively  small  number  of  observa¬ 
tions  will  be  available  for  a  certain  era  or  class  of  technology.  Here, 
even  e  near  perfect  information  system  cannot  increase  the  sample  size. 

Where  does  ell  this  leave  us?  On  the  one  hand,  a  strong  argument 
has  been  advanced  for  the  importance  of  an  appropriate  information  and 
data  base.  On  tha  other  hand,  trying  to  solve  the  problem  once-for-all 
does  not  seem  feasible,  at  least  in  a  general  sense.  Does  this  mean 
that  the  situation  is  an  Impasse? 

The  answer  is  "no."  The  problem  is  susceptible  to  reasonable  so¬ 
lution,  at  least  in  many  instances.  Numerous  alternatives  to  establish¬ 
ing  new  complete  enumeration  information  systems  may  be  considered. 

We  shall  now  outline  and  discuss  briefly  examples  of  a  few  of  the  ap¬ 
proaches  that  may  be  taken  to  help  solve  the  data  problem. 


Aa  will  be  pointed  out  later,  there  are  alternatives  to  complete 
enumerations  on  a  recurring  basis. 


Onu  vt>ry  Interesting  possibility  which  thfi  present  author  feels 
has  been  relatively  neglected  is  sampling*  or  sene thing  akin  to 
pling.  This  can  be  a  low  cost  way  of  obtaining  Information  that  may 
be  vary  useful  in  deriving  estimating  relationships  for  use  In  long- 
range  planning  studies. 

Suppose,  for  example,  that  the  coat  analyst  is  faced  with  the  prob¬ 
lem  of  developing  end-product  oriented  estimating  relationships  for 
some  functional  area  like  maintenance  or  supply  in  the  Department  of 
Defenae.  Suppose  further  that  the  existing  cost  Accounting  systems 
accumulate  historical  cost  data  in  categories  such  as  labor,  material, 
overhead,  etc.,  and  that  no  provision  is  made  for  identifications  so 
end-product  packages  of  military  capability  (e.g.,  weapon  nysUras). 
Conceivably  one  solution  would  be  to  overhaul  the  entire  formal  account¬ 
ing  system  to  accumulate  historical  cost  darn  in  the  desired  fora,  in 
addition  Co  the  existing  categories  needed  lor  purposes  of  functional 
management  of  the  maintenance  and  supply  activities.  Thia,  however, 
could  be  very  expensive,  and  considerable  time  would  have  to  elapse  to 
permit  design,  teat,  and  implement#? ion  of  the  new  accounting  syateai. 

An  alternative  would  be  to  select  a  few  representative  locations 
and  to  provide  for  an  ad  hoc  (temporary)  "ticketing"  ayatem  to  accumu- 
late  costs  in  terms  of  weapon  systems  for  a  relatively  short  pariod  of 
tlme--aay  a  mouth  or  two.  The  ad  hoc  arrangement  would  be  supplemen¬ 
tary  to- -and  hence  would  not  disturb--the  existing  formal  accounting 
system.  This  approach  has  been  uBed  on  numerous  occasions  in  the  p.  *s *■ , 
and  for  those  cases  known  to  the  author,  the  results  have  been  good- - 

at  leaat  for  the  purpose  of  deriving  estimating  relationships  for  long- 
* 

range  planning.  In  any  event,  sampling  procedures  seem  worthy  of 
- - 

The  author  has  conducted  simple  testa  in  several  instances  where 
complete  enumerations  were  available.  The  procedure  vaa  as  follows: 

Taka  tbs  complete  emnaration  as  s  date  base  end,  using  regression  anal 
ysls,  derive  an  estimating  relatlonshlp--say  C  -  St  +  BX.  then  take 
random  sables  of  IS  or  20  observations  from  the  complete  enumeration 
and  derive  similar  relationship*  on  the  basis  of  these  s maple  date  ba¬ 
ses.  Then  test  the  resulting  estimates  of  a  and  B  against  tbs  values 
obtained  from  the  complete  enumeration  to  see  if  there  is  s  signifi¬ 
cant  difference.  In  the  particular  cssas  examined  by  the  author,  most 
of  tbs  time  m  significant  difference  existed  (at  the  0,05  level)  be¬ 
tween  estimates  of  the  regression  coefficients  obtained  from  the  small 
samples  and  those  obtained  by  using  the  complete  enumeration  as  a  data 
base. 
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consideration  as  an  alternative  to  establishing  new  complete  enumera¬ 
tion  systems  across  the  board. 

Techniques  for  Assisting  In  Handling 
the  Small  Sample  Problem 

We  have  Indicated  that  the  cost  analyst  very  often  finds  himself 
confronted  with  small  samples.  Can  anything  be  done  to  help  ease  the 
problems  arising  from  having  to  use  data  bases  containing  only  a  small 
number  of  observations  from  the  historical  record?  Several  things  may 
be  done.  Let  us  consider  two  examples. 

The  first  is  an  extremely  simple  Idea,  but  in  6ome  instances  it 
can  help  a  great  deal.  Particularly  in  deriving  estimating  relation¬ 
ships  for  use  In  long-range  planning  studies,  the  cost  analyst  should 
not  necessarily  restrict  himself  to  the  historical  record  in  assembling 
his  data  baae.  In  many  caaea  he  should  seriously  consider  increasing 
the  number  of  observations  by  including  appropriate  data  points  based 
on  estimates  made  by  experts  for  the  very  near-term  future,  and/or  by 
taking  advantage  of  certain  kinds  of  qualitative  information. 

Suppose,  for  example,  we  have  only  four  data  points  available  from 
the  historical  record  (see  Fig.  9).  Suppose  further  that  the  analyst 
oust  darive  an  estimating  relationship  which  will  help  him  project  out 
bayond  the  range  of  the  historical  sample  (beyond  the  value  Xq  of  the 
explanatory  variable).  On  the  baais  at  the  four  data  points  alone,  It 
is  not  vary  clear  what  kind  of  relationship  between  C  and  X  should  be 
postulated.  For  example,  the  curves  AB  and  CD  in  Fig.  10  would  seem 
about  equally  plausible.  Here  is  a  case  where  the  cost  analyst  should 
probe  further  and  attempt  to  get  some  sort  of  additional  information 
(either  quantitative  or  qualitative)  to  help  him  make  an  informed  judgment, 

Suppose  that  in  our  hypothetical  example  the  cost  analyst,  upon 
further  exploration,  was  fortunate  enough  to  find  two  more  data  points 
In  the  form  of  estimates  for  the  near-term  future  made  by  reputable  ex- 
parts  in  the  field  under  consideration.  Upon  checking  out  the  methods 
used  to  make  these  estimates,  the  cost  analyst  decided  that  it  would 
be  appropriate  for  him  to  UBe  them  as  a  supplement  to  his  historical 
data  baae.  The  result  is  shown  in  Fig.  11.  This  would  tend  to  suggest 


Exp la hat 


sample  example — some  plausible  estimating  relationships 
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the  appropriateness  of  a  linear  hypothesis  as  a  basis  for  projecting 
out  to  the  vicinity  of  in  Fig.  11. 

Let  us  assume,  however,  that  our  cost  analyst  wanted  still  further 
substantiation--if  possible.  He  recalled  that  in  his  initial  search 
for  an  appropriate  explanatory  variable,  he  had  talked  to  some  engineers 
who  were  experts  in  designing  the  type  of  equipment  or  activity  under 
investigation  in  this  particular  case.  He  decided  to  consult  with  them 
again  in  the  hope  of  obtaining  some  thread  of  qualitative  evidence  which 
would  help  in  deciding  whether  to  accept  or  reject  the  linear  hypothesis. 
Upon  listening  to  the  engineers  discuss  the  structural  characteristics 
of  the  activity  under  consideration,  the  cost  analyst  became  convinced 
that  projections  for  large  values  of  the  explanatory  variable  X  should 
be  made  on  the  basis  of  a  linear  relationship  between  C  and  X. 

This  hypothetical  example  illustrates  two  points  about  how  one  can 
deal  with  very  small  samples:  (1)  Under  certain  conditions  the  size 
of  the  sample  can  be  increased  by  Judiciously  using  estimates  for  the 
near  future  as  supplements  to  the  historical  data  base;  (2)  it  may  be 
possible  to  use  qualitative  information  to  assist  in  deciding  about 
what  kind  of  estimating  relationship  is  most  appropriate. 

As  another  example,  let  us  consider  a  case  where  the  sample  is 
very  small  and  we  seek  to  gain  additional  information  by  lowering  the 
level  of  aggregation  one  notch. 

In  the  area  of  military  major  equipment  cost  analysis,  cost-quantity 

relationships  are  very  important.  As  the  cumulative  number  of  units 

* 

increases,  unit  cost  usually  declines.  Suppose  that  we  are  interested 
in  a  certain  type  of  aircraft  airframe  (call  it  X)  and  that  we  have 
only  three  data  points.  No  other  points  are  available  for  this 


For  a  thorough  treatment  of  cost-quantity  relationships,  see 
Harold  Asher,  Cost-Quantity  Relationships  in  the  Airframe  Industry. 
R-291  (Santa  Monica,  Calif.:  The  RAND  Corporation,  July  1,  1956). 
Cost-quantity  relationships  in  one  form  or  another  are  also  found  in 
other  areas.  For  example,  in  the  automobile  industry  unit  production 
costs  after  launching  a  n- w  model  are  considerably  higher  during  the 
earlier  part  of  the  production  run  than  they  are  later  in  the  model 
year.  These  "excess"  costs  are  called  "launching  costs"  in  the  auto¬ 
mobile  industry. 
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particular  airframe.  The  log- log  plot  of  the  data  base  is  shown  in 
Fig.  12. 

Assure  nov  that  the  cost  analysis  is  part  of  a  systems  analysis 
study  in  which  large  numbers  of  airframe  X  are  being  considered:  iCOO 
or  more.  Should  the  analyst  simply  assume  a  log-linear  relationship, 
connect  his  three  data  points,  and  extend  the  line  out  to  cumulative 
outputs  of  1000  or  more?  Most  probably  not.  An  experienced  analyst 
knows  all  too  well  the  dangers  of  mechanistic  extrapolation,  for  scaling 
factor  reasons  and  others  as  well. 

Since  in  our  hypothetical  example  the  sample  size  cannot  be 
increased,  what  can  be  done?  One  possibility  is  to  see  if  additional 
information  can  be  obtained  by  disaggregating.  Suppose  that  our  cost 
analyst  goes  back  to  the  original  data  source  and  finds  that  addi¬ 
tional  detail  is  in  fact  available.  He  obtains  a  breakdown  of  the  to¬ 
tal  airframe  in  terms  of  labor,  material,  and  overhead.  A  plot  of  these 
data  is  shown  in  Fig.  13.  This  slight  addition  to  the  data  base  imme¬ 
diately  provides  useful  insights  into  the  projection  problem.  If  we 

assume  log- linear  relationships  for  the  components  (labor,  material, 

* 

and  overhead),  it  is  obvious  that  on  the  basis  of  the  available  infor¬ 
mation  the  total  curve  cannot  be  log- linear  when  projected  out  to  large 
cumulative  unit  numbers  because  the  materials  curve  has  a  significantly 

IHc 

different  slope  than  the  labor  and  overhead  curves. 

If  the  curves  in  Fig.  13  are  extrapolated  out  to  cumulative  unit 
number  1000,  the  results  are  as  portrayed  in  Fig.  14.  Here  it  in  clear 
that  the  coat  analyst  has  benefited  from  the  Information  obtained  by 
disaggregating  one  level  in  the  data  base.  Merely  extrapolatl og  out 
to  cumulative  output  1000  on  the  basis  of  the  three  original  data 
points  no  longer  seems  appropriate.^  The  difference  between  the  two 


In  general  this  is  not  necessarily  a  good  assumption;  but  we  shall 
use  it  here  to  keep  the  example  simple.  The  argument  is  even  stronger 
if  the  component  curves  are  assumed  to  be  convex  on  logarithmic  grids. 

If  the  component  curves  are  linear  but  nonparallel,  the  total 
curve  (sum  of  the  components)  must  be  convex  on  logarithmic  grids  and 
must  approach  as  a  limit  the  flattest  of  the  component  curves.  (E.g., 
see  Asher,  o£.  clt . .  pp.  70-72.) 

The  difference  would  be  even  greater  if  the  component  curves  were 
assumed  to  be  convex. 


(Log-log  scale) 


Cumulative  Unit  Number 
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Fig.  13 — Dollars  per  pound  of  airframe  vs.  cumulative  unit  number 
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Fig.  14 — Dol 
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curve  a  increases  still  further  for  cumulative  unit  numbers  beyond  1000. 

This  example  illustrates  how  going  into  slightly  more  detail  can 
help  in  cases  where  the  cost  analyst  has  to  work  with  a  very  small 
sample.  A  word  of  caution  is  in  order,  however.  The  reader  should  not 
generalize  from  our  example  and  conclude  that  in  all  (or  even  moat)  in¬ 
stances  the  assembly  of  a  more  and  more  detailed  data  base  will.  In 
itself,  make  for  better  understanding  of  the  problem. 

The  Use  of  Experiments  to  Broaden  the  Data  Base 

Sometimes  the  cost  analyst  find9  that  in  a  given  problem  area 
there  is  simply  a  void  in  the  existing  formal  data  base.  This  is  likely 
to  be  the  case  when  the  planners  are  considering  new  proposals  for  dis¬ 
tant  future  programs  or  capabilities  requiring  major  equipments  and/or 
operational  concepts  markedly  different  from  those  of  the  past  and  the 
present. 

In  some  instances  the  existing  set  of  estimating  relationships 
can  be  used  to  conduct  simulations  which  will  furnish  a  first  approx¬ 
imation  to  the  cost  of  these  proposed  new  capabilities.  In  other  in¬ 
stances,  however,  the  cost  analyst  cannot  assume  that  the  structural 
parameters  in  the  existing  set  of  estimating  relationships  are  appro¬ 
priate  for  the  new  activities  being  considered.  He  must  therefore  de¬ 
velop  new  relationships,  or  devise  techniques  for  adjusting  the  present 
ones.  But  how  does  he  do  this  if  the  necessary  data  base  does  not  yet 
exist?  One  possibility  is  to  see  if  any  experiments  are  being  conduc¬ 
ted  pertaining  to  the  subject  at  hand;  and  if  not,  to  try  to  initiate 
such  an  experiment.  Let  us  consider  one  example  briefly. 

A  number  of  years  ago,  cost  analysts  were  confronted  with  the  task 
of  estimating  the  cost  of  the  first  generation  of  proposed  stainless 
steel  airframes  for  the  mid-1960s.  These  proposals  usually  required 
rather  extensive  use  of  stainless  steel  honeycomb  paneling,  the  produc¬ 
tion  of  which  would  Involve  a  significant  advance  in  the  manufacturing 


The  difference  is  only  about  1.5  dollars  per  pound  at  cumulative 
unit  number  1000.  At  cumulative  output  5000,  the  difference  between 
the  linear  projection  and  the  non-linear  total  curve  is  about  3  dollars 
per  pound. 
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stata  of  the  art.  The  historical  data  base  at  that  time  was,  of  course, 
confined  almost  entirely  to  the  experience  accumulated  in  producing 
aluminum  airframes,  and  little  vas  in  the  formal  records  about  the  fab¬ 
rication  costs  of  stainless  steel  honeycomb  pane la --particularly  large 
panel*. 

In  tha  process  of  talking  to  the  aerospace  industry  contractors 
regarding  the  problems  involved  in  fabricating  stainless  steel  struc¬ 
tures,  tha  cost  analysts  found  that  one  of  the  companies  was  conducting 
a  rather  elaborate  experiment,  A  apecial  shop  had  been  set  up  and  nu¬ 
merous  types  of  manufacturing  operations  were  being  performed  on  elumi- 
nisa,  stainless  steel,  and  titanium  structures ,  Taking  aluminum  sb  the 
base  case,  the  objective  of  the  experiment  was  to  determine  the  prob¬ 
able  incrementcl  labor  coats  involved  In  working  the  other  two  materials 
for  a  representative  sample  of  varioua  types  of  manufacturing  operations. 
Armed  with  these  types  of  data  from  the  experiment,  the  cost  analysts 
were  than  in  a  position  to  devise  techniques  for  adjusting  the  histor¬ 
ical  data  baas  (aluminum  experience)  so  that  it  would  be  more  appropri¬ 
ate  for  dealing  with  tha  stainless  steal  airframe  problem. 

In  visits  to  still  other  contractors'  plants,  tha  cost  analysts 
found  that  several  ware  experimenting  with  the  construction  of  stain¬ 
less  steel  honeycomb  paneling.  In  sessions  with  tha  people  conducting 
these  operations  tha  coat  analysts  obtained  a  wealth  of  information 
(both  quantitative  and  qualitative)  about  how  honeycomb  coat  stight  vary 
with  core  call  airs  and  shape,  shape  and  alze  of  the  penal,  masher  of 
penal  Inserts,  and  the  like.  As  a  result,  they  were  able  to  treat  pan¬ 
els  as  a  special  cost  analysis  problem  and  hence  to  improve  consider¬ 
ably  their  ability  to  estimate  the  cost  of  stainless  steel  alrf reams. 

Tha  expenditure  of  time  and  travel  budget  on  field  work  paid  off  well. 

SuMarr  Comment 

Rather  typically  cost  analysts  supporting  a  systems  analysis  ac¬ 
tivity  spend  at  least  half  their  time  struggling  with  the  date  and  in¬ 
formation  problem.  In  this  section  we  have  tried  to  convey  some  flavor 
of  the  total  problem  and  some  notion  of  the  types  of  techniques  that 
may  be  employed  to  solve  it.  Basically  what  is  required  is  ingenuity, 
peralatence,  and  just  plain  hard  work. 
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Systems  Analysis  foms  the  cantral  core  of  a  program  budgeting 
activity.  A  vitally  important  part  of  systems  analysis  is  a  cost  anal* 
yals  capability  to  generate  estimates  of  the  resource  impact  of  alter" 
native  courses  of  action  being  considered  for  the  distant  future. 

Soon  of  the  principal  characteristics  of  a  systems  cost  analysis 
capability  are: 

1.  An  explicit  relationship  between  inputs  and  outputs,  with  a 
strong  emphasis  on  assessing  the  economic  cost  of  alternative 
future  output-oriented  program  packages. 

2.  Explicit  treatment  of  uncertainty. 

3.  Provision  for  dealing  with  scaling  considerations. 

4.  Explicit  treatment  of  problems  associated  with  time. 

5.  A  recognition  of  the  importance  of  sensitivity  analysis,  con¬ 
tingency  analysis,  and  a  fort iorl  argument. 

6.  Allocation  of  a  substantial  amount  of  time  and  effort  to  the 
continuous  development  and  maintenance  of  an  appropriate  data 
basa. 

Establishing  and  maintaining  a  cost  analysis  capability  to  support 
systems  analysis  studies  involves  numerous  difficulties.  One  of  the 
most  troublesome  is  the  date  beee  problem. 

Solution  to  parts  of  the  dsta  problem  may  be  through  major  over¬ 
haul  of  present  formal  Information  systems  and  through  the  establish¬ 
ment  of  new  complete  enumeration  systems.  This,  however,  does  not 
appear  feasible  as  a  general  solutlon--at  least  in  the  foreseeable  future. 

Short  of  such  major  efforts  are  numerous  alternative  possibilities. 
Some  ex  maples  ere; 


1.  use  of  seiapling  techniques  on  an  ad  hoc  basis. 

2.  Supplementing  the  existing  historical  data  base  by  including 
estimated  data  points  for  the  near  future. 

3.  Statistical  manipulation  of  the  existing  data  base. 

4.  Obtaining  additional  information  by  conducting  experiments. 
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